Guidelines for Good Practice in Bayesian Network Modelling

Bayesian networks (BNs) are used increasingly to model environmental systems, for reasons including their ability to: integrate multiple issues and system components; utilise information from different sources; and handle missing data and uncertainty. For a model to be of value in generating and sharing knowledge or providing decision support, it must be built using good modelling practice. This paper provides such guidelines to developing and evaluating Bayesian network models of environmental systems. The guidelines entail clearly defining the model objectives and scope, and using a conceptual model of the system to form the structure of the BN, which should be parsimonious yet capture all key components and processes. After the states and conditional probabilities of all variables are defined, the BN should be evaluated by sensitivity analysis, expert review and testing with cases. All the assumptions, uncertainties, descriptions and reasoning for each node and linkage, data and information sources, and evaluation results must be clearly documented. Following these minimum standards will help ensure the modelling process and the model itself is transparent, credible and robust, within its given limitations. Introduction Environmental systems are inherently complex and there is often a high degree of uncertainty of the interactions of system components. While traditional statistical modelling approaches can be used for some models of single components or processes within the system (e.g. rainfall-runoff models, hydraulic models), integrated modelling approaches are often required for whole-of-system models or models incorporating multiple system components. Commonly used integration methods include Bayesian networks (BNs), system dynamics, coupled component models, agent-based models and expert systems. These methods vary in their knowledge and data requirements, technical requirements, treatment of uncertainty, and application suitability (Jakeman et al. 2007). This paper focuses on Bayesian networks, as it is an approach considered highly suitable for environmental problems due to its ability to integrate multiple issues, interactions and outcomes and investigate tradeoffs. Furthermore, Bayesian networks are apt at utilising data and knowledge from different sources, and handling missing data and uncertainty. Bayesian networks are based on a relatively simple causal graphical structure, which means it can be built without highly technical modelling skills and it can also be understood by non-technical users and stakeholders. This is a very valuable feature of Bayesian networks, particularly in the context of natural resource management which (ideally) involves interdisciplinary and participatory processes. There is great benefit in the use of modelling as an approach to understanding and supporting decisions on environmental systems. However, for a model to be of value, good practice in its construction, testing and application is essential, as is awareness of the purposes, capabilities and limitations of the modelling approach. Without this, there is a risk of the model user misinterpreting or misusing model outputs, and drawing invalid conclusions (Jakeman et al. 2006). For the model user to be aware of the modelling objectives, assumptions and limitations, the modeller needs clear reporting protocols. Poor modelling practice reduces the credibility of the model and can lead to the model capabilities being ‘oversold’, potentially causing poor decisions to be made based on models, or where model transparency and testing has not been completed, users mistrusting models and their outputs (Refsgaard and Henriksen 2004). Consequently, guidelines for good modelling practice that create standards to help ensure the development and application of credible and purposeful models are essential. Several authors have developed modelling guidelines (Refsgaard and Henriksen 2004, Jakeman et al. 2006, Crout et al. 2008), where the key components for good practice include:  Clearly defining model purpose and the assumptions underlying the model  Thorough evaluation of the model and its results  Transparent reporting of the whole modelling process, including its formulation, parameterisation, implementation and evaluation Good modelling practice will result in better understanding of the development and application of models; this benefits not only the modelling community but also model users who employ the models for improving knowledge of the system or decision making. The objective of this paper is to introduce guidelines to developing and evaluating Bayesian network models of environmental systems. As with models in general, there is a need for quality assurance standards in developing and applying Bayesian network models. Bayesian network protocols have been published by Cain (2001) and Marcot (2006). Cain (2001) provided guidelines to using BNs for supporting planning and management of natural resources, with a large emphasis on facilitating stakeholder consultation. In the context of natural resources management, stakeholder consultation is seen as essential to ensuring that the management plan is followed through and implemented (Cain 2001). Marcot (2006) developed guidelines for Bayesian networks applied to wildlife and ecological assessment, with the steps to developing and updating the BNs described at three model levels: alpha, beta and gamma. The alpha-level model is the initial functioning BN, suitable only for internal use and review. The BN is considered a beta-level model after formal peer review and revision is conducted. The gamma-level or final application model, is created by further testing, calibrating, validating and updating the beta-level model (Marcot et al. 2006). This paper will explore the development process of Bayesian network models, following the generic guidelines for good modelling practice outlined by Jakeman et al. (2006). These guidelines consist of ten iterative steps (Jakeman et al. 2006): 1. Define model purpose 2. Specify modelling context (scope and resources) 3. Conceptualise the system, specify data and other prior knowledge 4. Select model features and families 5. Decide how to find model structure and parameter values 6. Select estimation performance criteria and technique 7. Identify model structure and parameters 8. Conditional verification and diagnostic testing 9. Quantify uncertainty 10. Model evaluation and testing The paper is intended to serve a wide readership. It is envisaged that adhering to the proposed guidelines will enhance their quality and value in generating and sharing knowledge on environmental systems and providing advice on their management. Bayesian networks in Natural Resources Management In BN models, the studied system is represented as a complex network of interactions from primary cause to final outcome, with all causal assumptions made explicit (Borsuk et al. 2006). Evidence is entered into the model by substituting the a priori beliefs of one or more nodes (variables) with observation or scenario values. Through belief propagation using Bayes’ Theorem, the a priori probabilities of the other nodes are updated. This belief propagation enables BNs to be used for diagnostic (‘bottom-up’ reasoning) or explanatory purposes (‘top-down’ reasoning) (Castelletti and Soncini-Sessa 2007). So unlike black-box models, such as neural networks, BN users can find out the reasoning behind the model outputs as interactions between variables are clearly displayed. This not only provides clarity to users, but also promotes system learning and increases the transparency of management decisions. BNs can effectively integrate information from a range of sources and are also able to integrate submodels, even those representing different scales (Borusk et al. 2004). BNs can be used to predict future states/events even when data is partial or uncertain (Park et al. 2005). This is a huge advantage over many other traditional statistical models which often rely on large amounts of empirical data to be built (Marcot et al. 2006). However, as with all modelling approaches, BNs are limited in some respects. BNs are Directed Acyclic Graphs, so they cannot represent feedback loops, which often occur in nature. Also, BNs generally represent static relationships over given temporal scales, although some software packages can handle dynamic models by representing each time slice with a separate network (Kjærulff 1995). BNs can also be useful to decision-makers as the model can be used to investigate tradeoffs. For example, BNs can be used to test and compare the forecasted system response to alternative policy or management options (e.g. Ticehurst et al. 2007), which can help inform managers on which scenario is likely to produce the optimal outcome based on the information given to the network. More details about the advantages and limitations of BNs in environmental modelling can be found in Castelletti and Soncini-Sessa (2007) and Uusitalo (2007). In the environmental domain, BNs are often used to integrate information about the factors influencing certain aspects of a species, community or system component, to aid management. Examples of such BNs include habitat and population viability models of at-risk fish and wildlife species in the Columbia River basin (Marcot et al. 2001), a dynamic, age-structured population model of brown trout in Swiss Rivers, for assessing the relative influence of different stress factors (e.g. water quality, habitat conditions, stocking practices, flood frequency) to indicate the type of management actions that would be most effective in protecting their populations (Borsuk et al. 2006) and an eutrophication model for the Neuse estuary, North Carolina, developed to quantify the relationship between nitrogen loading and other relevant variables (e.g. shellfish population, size and frequency of algal blooms, fish kill) and assist decision makers who were considering new legislation on total maximum daily load of nitrogen (Borusk et al. 2004). BNs can effectively integrate physical, social, ecological and economic components of a system into a model. Accordingly they have been applied as integrated models used as decision support tools for testing the impact of various management strategies (pricing, awareness-education, grey water reuse and leak) on domestic water consumption in the Loddon catchment, England (Bromley et al. 2004), assessing the ecological impacts of salinity management scenarios for the Litter River Catchment, Macquarie River basin, NSW (Sadoddin et al. 2005) and exploring the impact of various climate change scenarios on natural resource condition targets (e.g. Red Gum growth rate, bird breeding event, Macquarie Marshes water quality) in the Central West region, NSW (Tighe et al. 2007). BNs can also be applied as risk assessment models, as in Pollino et al. (2008) where a BN was developed to predict the impact of mine-derived heavy metals to the environment and human health in the Ok Tedi and Fly River, Papua New Guinea. In Pollino et al. (2007a), BNs were used as a modelling framework to examine conflicting hypotheses on the main causes of dieback in the Swamp Gum in the Yellingbo Nature Conservation Reserve, Victoria, and make recommendations for future monitoring and research. As these examples demonstrate, there is an enormous scope for the possible applications of BNs in natural resources management. Guidelines to good practice in Bayesian network modelling 1) Define the model purpose Clearly defining the model purpose and scope is the first key component of most modelling guidelines (Cain 2001, Jakeman et al. 2006, Crout et al. 2008). It is important that the objectives of the modelling exercise are clear from the beginning, to ensure that the network is built to fulfil the right purpose and captures all the relevant ideas. The model purpose influences many of the choices in the modelling development process, including what variables or information to include, the level of detail required, the complexity of the structure, and the scales considered. Model purpose also determines the role of uncertainty and how the uncertainty should be handled (Brugnach et al. 2008). Furthermore, purpose drives the model evaluation process; without specifying the purpose its success cannot be assessed (Crout et al. 2008). Motives for developing and applying Bayesian network models can include:  Improving system understanding